Method for selecting a target in an automated video tracking system

ABSTRACT

A method for selecting a target in an automated video tracking system. The method including the steps of: displaying video image data of a scene having a target to be tracked; freezing the video image data to display a frame thereof; and selecting a desired target in the frame of the video image data. Also provided is an apparatus for selecting a target in an automated video tracking system and an automated video tracking system employing such methods and apparatus.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates generally to video tracking systems and,more particularly, to a method for selecting a target in an automatedvideo tracking system.

2. Prior Art

For purposes of this disclosure, automated tracking is defined as theautomatic control of the Pan, Tilt and Zoom (PTZ) motors of a movablePTZ camera so as to keep the camera view centered on a designated,moving target. Automated tracking as defined is used in a number ofdifferent applications areas, such as surveillance and securitymonitoring. In this area, the target is usually a human.

Automated tracking systems typically have several parts, targetselection, model generation, and camera control. A target needs to beselected for tracking. This can be via an operator or via an automatedmotion detection module or other intruder detection system. An internal“model” of the appearance of the target is necessary to allow thetracking system to find the target in subsequent images. A camera motioncontrol model is necessary to determine how to move the camera to keepthe target in the center of the field of view.

The present disclosure relates to the problem of target selection.Identification of potential tracking candidates (i.e., a desired target)in a video scene is typically not part of the function of an automatedtracking system. For instance, in the area of surveillance, targetselection requires a lot of background knowledge about the objective ofany surveillance application. What looks “suspicious” in onesurveillance application, e.g. a retail store, may not look suspiciousin another, e.g. a parking lot.

In some applications, any source of motion is suspicious, e.g.,monitoring a warehouse at night. In that case, an intrusion detectionsensor, or a motion sensor, could be used to designate a target fortracking. A more sophisticated automatic monitoring system, could beused to designate targets for certain other applications, as long as therules to select targets can be clearly enumerated and implemented.However, in general, it is expected that a human operator will indicatethe target to the tracking system.

When an operator designates a person in the video image as the trackingsystem's target, there is a subtle difference in meaning between theoperator's and the tracking system's concept of the target. The operatoris designating a person as the target, however, the tracking system issimply accepting a region of the image as the target. Because of this,the operator may not be overly fussy about what part of the person hepicks, since after all, its clear to any (human) observer which personhe or she selects. Furthermore, the tracking system will form a targetmodel based on exactly what image region the operator selected. As ithas no independent knowledge of the desired target it cannot generalizebeyond what it is told. Further problems arise because the operator isselecting a moving target, therefore he or she may select a bad target,a portion of a target, or possibly even an empty space.

SUMMARY OF THE INVENTION

Therefore it is an object of the present invention to provide a methodand apparatus for selecting a target in a video tracking system whichresolves the problems with the prior art video tracking systems.

Accordingly, a method for selecting a target in an automated videotracking system is provided. The method comprises the steps of:displaying video image data of a scene having a target to be tracked;freezing the video image data to display a frame thereof; and selectinga desired target in the frame of the video image data. Preferably, themethod further comprises the steps of: unfreezing the video image dataafter the selecting step; and tracking the selected desired target inthe unfrozen video image data.

The selecting step preferably comprises drawing a shape, such as arectangle around at least a portion of the desired target with a inputdevice, such as a computer mouse. Alternatively, the selecting stepcomprises indicating a region in which the desired target resides withan input device, such as using a computer mouse to click on the region.

Also provided is an apparatus for selecting a target in an automatedvideo tracking system. The apparatus comprises: a camera for providingvideo image data of a scene having a desired target to be tracked; adisplay for displaying the video image data; means for freezing thevideo image data to display a frame thereof; and an input device forselecting the desired target in the frame of the video image data.Preferably, the apparatus further comprises: means for unfreezing thevideo image data after the desired target is selected; and an automatedvideo tracking system having a controller for controlling the camera totrack the selected desired target in the unfrozen video image data.

The input device preferably comprises means for drawing a shape, such asa rectangle around at least a portion of the desired target.Alternatively, the input device comprises means for indicating a regionin which the desired target resides, such as by clicking on the region.Preferably, the input device is a computer mouse.

Also provided is a video tracking system comprising: a camera forproviding video image data of a scene having a desired target to betracked; a display for displaying the video image data; means forfreezing the video image data to display a frame thereof; an inputdevice for selecting the desired target in the frame of the video imagedata; means for unfreezing the video image data after the desired targetis selected; and a controller for controlling the camera to track theselected desired target in the unfrozen video image data.

Preferably, the input device comprises means for drawing a shape, suchas a rectangle, around at least a portion of the desired target.Alternatively, the input device comprises means for indicating a regionin which the desired target resides, such as by clicking on the region.Preferably, the input device is a computer mouse.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other features, aspects, and advantages of the apparatus andmethods of the present invention will become better understood withregard to the following description, appended claims, and accompanyingdrawings where:

FIG. 1 illustrates a preferred video tracking system of the presentinvention.

FIG. 2 illustrates a preferred method for selecting a target in thevideo tracking system of FIG. 1.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

Although this invention is applicable to numerous and various types ofvideo tracking systems, it has been found particularly useful in theenvironment of security systems. Therefore, without limiting theapplicability of the invention to security systems, the invention willbe described in such environment. Those skilled in the art willappreciate that the methods and apparatus of the present invention alsohave usefulness in such areas as videoconferencing and multi-nodalinterfaces for consumer devices.

Referring now to FIG. 1, there is illustrated a preferred implementationof the video tracking system of the present invention, generallyreferred to by reference numeral 100. The apparatus 100 comprises acamera 102 for providing video image data of a scene 104 having adesired target 106 to be tracked. The camera 102 is preferably a PTZcamera having PTZ motors 108 for controlling the pan, tilt and zoom ofthe camera 102. Such camera and motors for their control are well knownin the art.

The apparatus 100 further includes a display, such as a computer monitor110 for displaying the video image data of the scene 104 from the camera102. The computer monitor's display is generally referred to herein byreference numeral 104 a. An input device is used to select the desiredtarget 106 a in the video image data Reference numeral 106 is usedherein to indicate the actual target while reference numeral 106 aindicates the image of the target as displayed on the monitor 110. Theinput device is preferably a computer mouse 112 connected to a computerprocessor 114. However, any input device that is capable of selecting atarget in the video image display can be utilized without departing fromthe scope or spirit of the present invention. Such other input devicescan be a touchpad, touchscreen, touchpen, or even a keyboard 113connected to the computer 114. The computer mouse 112 preferablyindicates the target in the video image display by drawing a shapearound at least a portion of the target 106 a. Preferably, the shape isa rectangle, however, any other shape can also be used, such as anellipse. Alternatively, other methods can be utilized for selecting atarget, such as merely clicking on a region of the target 106 a in thevideo image display and using a region-filling algorithm, such as iswell know in the prior art, to locate the target region.

Once the operator has selected the target 106 a, a tracking system 116generates a model of the target 106 a that can be used to locate thetarget 106 a in successive frames of the video image data. Such trackingsystems are well known in the art. Although shown separately in FIG. 1,the tracking system 116 is preferably implemented by software containedon a peripheral device (not shown) in the computer processor 114.

Typically, there are two interconnected ways in which the model is used,to distinguish the target 106 a from the background scene or todistinguish the target 106 a from other occluding targets. Because themodel is gathered from the video image, it is clear that it can onlycontain information about appearance. This gives rise to the mostimportant constraint limiting the behavior of automated tracking,referred to as the appearance constraint. In general, a target can onlybe successfully tracked if its appearance distinguishes it from otherpotential targets. In other words, if the target does not have somethingunique about its appearance within the kind of visual environments inwhich the tracker is operating, then it is not possible to build aunique “model” for that target.

Once a model of the target 106 a is generated, a controller 118 is theninstructed by the tracking system to control the PTZ motors 108 to movethe camera to keep the selected target 106 centered in the field of viewof the camera 102. Such controllers are also well known in the art. Thecontroller 118, like the tracking system 116, is preferably implementedby software contained on a peripheral device on the computer 114. Twogeneral approaches for controlling the camera 102 that are widely usedin the prior art include a discrete approach in which the camera 102 ismoved from time to time to keep the target 106 centered and a continuousapproach in which the camera 102 is moved to keep the target 106continuously centered.

Whichever approach is used to control the camera 102, the results ofwhich are fed to the controller 118 which determines the amount ofcamera movement necessary to keep the target 106 centered in the fieldof view of the camera 102. The controller 118 outputs an appropriatesignal to the PTZ motors 108 of the camera 102 to carry out itsinstructions to keep the target 106 centered in the camera's 102 fieldof view. Such systems are well known in the art.

A further aspect of the apparatus for selecting the target will now bedescribed with reference to FIG. 2. FIG. 2 illustrates in more detailthe monitor display 104 a of FIG. 1. The monitor display 104 a is shownas having a windows environment. A portion of the video image display,referred to as a main video display 200, displays the video image datafrom the camera 102.

The target 106 a is chosen in the main video display 200 as discussedabove. However, to ensure the proper selection of the target 106 a, theoperator has the option of freezing the video image data to display aframe of such in the main video display portion 200 prior to selectingthe target 106 a. The video image data can be frozen by any number ofmeans, preferably, by clicking on a freeze button 202 provided on themonitor display 104 a. Alternatively, the operator can click on the mainvideo display 200 to freeze the video.

After freezing the main video display 200, the operator can select atarget 106 a as discussed above, preferably by drawing rectangle 204around at least a portion of the target 106 a. Preferably, the monitordisplay 104 a also includes a target display 206 which displays theselected target 106 a. In a typical Microsoft Windows application, therectangle 204 is drawn by placing the mouse pointer at the upper rightof the target region, right clicking, and while holding the right buttondown, moving the mouse pointer to the lower left of the target region,and releasing the right mouse button. In selecting the target region(rectangle 204) the operator should try to include as little of thebackground as possible. When the button is released, a rectangle 204will be drawn around the target region on the main video display 200,and the image region around the target will be copied to the targetdisplay 206.

Thus, if the operator is not satisfied with the selected target, he orshe can select another one. After a satisfactory target 106 a isselected, the operator unfreezes the main video display 200, by use ofanother button provided on the monitor display 104 a or preferably byuse of button 202. The tracking system then tracks the selected targeteither automatically after a satisfactory target is selected or manuallyby indicating such to the tracking system 116, such as by a button 208provided on the monitor display 104 a.

As discussed above, when the target 106 a is selected, a computer modelis build to represent the appearance of that target 106 a. Duringtracking of the target 106 a, whenever the tracker finds a part of theimage that matches to the target model, it preferably computes a numberwhich represents how well the target 106 a matches the model. Thisnumber can vary for example from 0% match to 100% match. Where the 100%indicates that the target matches the model completely. This value iscalled the confidence value. An indicator can be added to the monitordisplay 104 a of the tracker that indicates the confidence level 210.Furthermore, a control can also be added to the monitor display 104 afor indicating a threshold value 212 for the confidence. Thus, shouldthe model match the target with less than this threshold value, thetracker may warn the operator or stop tracking and consider that thetarget is lost.

While there has been shown and described what is considered to bepreferred embodiments of the invention, it will, of course, beunderstood that various modifications and changes in form or detailcould readily be made without departing from the spirit of theinvention. It is therefore intended that the invention be not limited tothe exact forms described and illustrated, but should be constructed tocover all modifications that may fall within the scope of the appendedclaims.

What is claimed is:
 1. A method for selecting a target in an automated video tracking system, the method comprising the steps of: displaying video image data of a scene having a target to be tracked; freezing the video image data of the scene having the target to be tracked to display a frame thereof; and selecting at least distinguishing image characteristics of the target relative to the entire scene in the frame of the frozen video image data.
 2. The method of claim 1, further comprising the steps of: unfreezing the video image data after the selecting step; and tracking the selected desired target in the unfrozen video image data.
 3. The method of claim 1, wherein the selecting step comprises drawing a shape around at least a distinguishing portion of the desired target with an input device.
 4. The method of claim 3, wherein the input device is a computer mouse and the shape is a rectangle.
 5. The method of claim 1, wherein the selecting step comprises indicating a region in which the desired target resides with an input device.
 6. The method of claim 5, wherein the input device is a computer mouse and the indicating step comprises clicking on the region.
 7. An apparatus for selecting target in an automated video tracking system, the apparatus comprising: a camera for providing video image data of a scene having a desired target to be tracked; a display for displaying the video image data; means for freezing the video image data of the scene having a target to be tracked to display a frame thereof; and an input device for selecting at least distinguishing image characteristics of the desired target relative to the entire scene in the frame of the frozen video image data.
 8. The apparatus of claim 7, further comprising: means for unfreezing the video image data after the image characteristics of the desired target are selected; and an automated video tracking system having a controller for controlling the camera to track the selected desired target in the unfrozen video image data.
 9. The apparatus of claim 7, wherein the input device comprises means for drawing a shape around at least a portion of the desired target containing the distinguishing image characteristics.
 10. The apparatus of claim 9, wherein the input device is a computer mouse and the shape is a rectangle.
 11. The apparatus of claim 7, wherein the input device comprises means for indicating a region in which the desired target resides.
 12. The apparatus of claim 11, wherein the input device is a computer mouse and the means for indicating comprises clicking on the region.
 13. A video tracking system comprising: a camera for providing video image data of a scene having a desired target to be tracked; a display for displaying the video image data; means for freezing the video image data of the scene having a target to be tracked to display a frame thereof; an input device for selecting at least distinguishing image characteristics of the desired target relative to the entire scene in the frame of the frozen video image data; means for unfreezing the video image data after the image characteristics of the desired target are selected; and a controller for controlling the camera to track the selected desired target in the unfrozen video image data.
 14. The video tracking system of claim 13, wherein the input device comprises means for drawing a shape around at least a portion of the desired target containing the distinguishing image characteristics.
 15. The video tracking system of claim 14, wherein the input device is a computer mouse and the shape is a rectangle.
 16. The video tracking system of claim 13, wherein the input device comprises means for indicating a region in which the desired target resides.
 17. The video tracking system of claim 16, wherein the input device is a computer mouse and the means for indicating comprises clicking on the region. 